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High  rates  of  attrition  among  students  In  Undergraduate  Navigator  Training  (UNT)  continue  to 
be  a  major  concern  for  Air  Training  Command.  In  an  effort  to  address  this  problem,  research  was 
conducted  by  the  Air  Force  Human  Resources  Laboratory  to  reduce  attrition  by  improving  the  method 
by  which  candidates  are  selected.  Currently,  UNT  students  are  chosen  primarily  on  the  basis  of 
their  scores  on  the  Navigator-Technical  (Nav-Tech)  composite  of  the  Air  Force  Officer  Qualifying 
Test  (AFOQT).  The  present  effort  sought  to  determine  the  extent  to  which  scores  on  an 
experimental  selection  battery,  known  as  the  Basic  Navigator  Battery  (BNB),  added  to  the  quality 
of  selection  screening  provided  solely  by  the  Nav-Tech  composite  score. 

Three  selection  models  were  evaluated  against  four  UNT  performance  standards.  The  three 
models  examined  consisted  of  AFOQT  Nav-Tech  composite  score  only  (Model  I),  AFOQT  Nav-Tech 
composite  score  plus  other  AFOQT  composite  scores  (Model  II),  and  AFOQT  Nav-Tech  composite  score, 
other  AFOQT  composite  scores  and  BNB  subtest  scores  (Model  III).  The  four  measures  of  UNT 
performance  were  training  outcome  (graduatlon/ellminatlon),  average  classroom  lesson  score, 
average  simulator  lesson  score,  and  average  flying  lesson  score. 

Model  III  was  found  to  have  the  highest  predictive  accuracy  for  all  four  performance 
standards.  Compared  to  Models  I  and  II,  Model  III  Identified  more  unqualified  candidates.  This 
model  was  also  particularly  good  In  predicting  classroom  and  simulator  performance,  suggesting  It 
may  be  useful  as  a  diagnostic  aid  to  identify  students  who  may  need  remedial  help  during  training. 

The  results  of  this  effort  provided  preliminary  evidence  of  the  usefulness  of  the  BNB  In 
reducing  UNT  attrition  through  Improved  selection.  Further  research  Is  necessary,  however,  to 
obtain  better  estimates  of  the  stability  and  validity  of  this  test  and  to  develop  new  forms  of 
the  BNB  before  It  could  be  used  operationally. 
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This  work  was  performed  under  Project  7719,  Air  Force  Personnel  System  Development 
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Utilization;  Task  771918,  Selection  and  Classification  Instruments  for  Officer  Personnel 
Programs,  and  responds  to  Request  for  Personnel  Research  (RPR)  No.  78-12,  Selection  for 
Undergraduate  Navigator  Training,  Issued  by  Air  Training  Command.  The  authors  wish  to 
express  their  appreciation  to  Ben  Roach,  Debbie  Rogers,  and  Gary  McDaniel  for  their 
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BASIC  NAVIGATOR  BATTERY:  AN  EXPERIMENTAL  SELECTION 
COMPOSITE  FOR  UNDERGRADUATE  NAVIGATOR  TRAINING 


I.  INTRODUCTION 

Unacceptably  high  rates  of  attrition  among  students  in  the  Undergraduate  Navigator  Training 
(UNT)  Program  have  been  a  continuing  concern  for  the  Air  Training  Coimnand  (ATC).  Loss  of 
students  in  UNT  represents  a  substantial  cost  to  the  Air  Force  both  in  terms  of  the  waste  of 
training  resources  and  the  subsequent  difficulty  in  meeting  operational  manpower  needs.  While 
retention  of  all  entering  trainees  is  virtually  Impossible  due  to  a  variety  of  factors  beyond  the 

control  of  the  Air  Force,  lower  rates  of  attrition  may  be  achieved  by  improving  the 

screening/selection  of  trainees,  and  designing  curriculum  and  Instructional  methods  to  optimize 
student  learning.  This  report  focuses  on  the  former  of  these  two  approaches  to  minimizing  the 
loss  of  students  In  UNT. 

The  most  obvious  method  for  reducing  attrition  among  UNT  trainees  Is  through  the  Initial 
selection  of  highly  qualified  candidates.  Since  World  War  II,  trainees  have  been  selected 

primarily  on  the  basis  of  their  scores  on  the  Navigator-Technical  (Nav-Tech)  composite  of  the  Air 
Force  Officer  Qualifying  Test  (AFOQT).  The  validity  of  this  paper-and-pencil  test  for  predicting 
success  (i.e.,  passing)  In  UNT  has  been  supported  by  previous  research  (Miller,  1966;  Valentine  & 
Creager,  1961). 

In  spite  of  the  significant  predictive  validity  of  the  AFOQT,  continuously  high  trainee 

attrition  rates  have  provided  the  Impetus  for  research  to  identify  supplemental  and/or 
alternative  selection  Instruments.  Based  on  their  multi-faceted  investigation  of  the  causes  of 
turnover  In  UNT,  DeVries,  Yakimo,  Curtin,  and  McKenzie  (1975)  reconmended  the  implementation  of  a 
selection  system  which  utilized  AFOQT  scores,  as  well  as  personality.  Interest,  and  cognitive 
ability  measures.  Subsequently,  Valentine  (1977)  conducted  a  study  designed  to  reassess  the 
predictive  validity  of  the  AFOQT  following  changes  in  the  UNT  curriculum,  and  also  to  examine  the 
potential  validity  of  an  experimental  selection  battery  consisting  of  personality.  Interest, 
biographical,  attitude,  and  cognitive  ability  measures.  Improvements  In  the  predictive  validity 
of  the  AFOQT  were  achieved  with  some  of  the  cognitive  ability  measures,  but  not  with  aqy  of  the 
noncognltlve  measures.  On  the  basis  of  this  research,  Valentine  reconmended  a  revision  of  the 
Nav-Tech  composite  of  the  AFOQT  to  Incorporate  those  measures  which  were  significantly  related  to 
success  in  UNT. 

The  purpose  of  the  present  effort  was  to  examine  the  potential  usefulness  of  an  experimental 
navigator  selection  battery,  the  Basic  Navigator  Battery  (BNB),  in  increasing  the  predictive 
validity  of  the  current  navigator  selection  system.  Specifically,  this  effort  attempted  to 
determine  which,  if  any,  of  the  subtests  of  the  BNB  added  significantly  to  the  overall  predictive 
validity  of  the  Nav-Tech  composite  of  the  AFOQT.  The  present  study  represented  a  significant 
departure  from  previous  Investigations  of  navigator  selection  In  that,  in  addition  to  the 
standard  pass/fall  criterion  of  success  in  training,  lesson  grades  for  each  phase  of  UNT  were 
also  used  as  criterion  measures.  Use  of  these  lesson  grades  provides  a  more  detailed  analysis  of 
the  relationships  between  the  predictors  of  Interest  (i.e.,  AFOQT,  BNB)  and  specific  phases  of 
UNT  training. 


II.  APPROACH 

Subjects 

Air  Force  officers  entering  UNT  (course  N-V6A-F)  at  Mather  AFB,  California,  during  the  1980 
calendar  year  (CY80)  served  as  subjects.  This  sample  was  comprised  of  UNT  classes  80-18  through 


81-17.  Of  the  800  officers  who  went  through  the  program  during  this  period,  relatively  complete 
data  were  available  for  544,  who  comprised  the  sample  analyzed  In  this  report. 

Predictor  Variables 

Two  sets  of  predictor  variables  were  examined.  The  first  consisted  of  the  five  composites 
(Pilot,  Navigator-Technical,  Academic  Aptitude,  Verbal,  and  Quantitative)  of  the  AFQQT.  Conmon 
metric  percentile  scores  (Roach  S  Rogers,  1982)  for  these  subtests  were  obtained  from  historical 
personnel  records.  The  second  set  of  predictors  was  the  percentage  scores  from  the  five  subtests 
of  the  BNB.  The  BNB  was  administered  to  all  subjects  1  week  prior  to  the  beginning  of  their 
assigned  training  class.  The  BNB  subtests  were  as  follows: 

Pre-Nav  (PE7935)  --  a  40-Item,  multiple-choice  test  measuring  mathematical  reasoning 
ability  within  the  context  of  navigational  problem  solving.  This  test  was  originally 
designed  as  a  diagnostic  Instrument  for  Identifying  students  in  need  of  remedial 
mathematics  training. 

Information  Processing  { PE7935)  —  a  30-Item  test  of  perceptual  reasoning,  divided 
equally  between  hidden  figures  Items  and  figure  analogy  Items.  Hidden  figures  Is  a 
template-matching  task  that  requires  the  subject  to  locate  a  simple  geometric  figure 
embedded  within  a  complex  drawing.  Figure  analogies  Is  a  test  of  symbolic  logic  in 
which  the  subject  must  first  discern  the  logic  of  a  relationship  { 1 . e. ,  shape,  size, 
axis  rotation)  between  a  pair  of  sample  geometric  figures,  and  then  apply  this  logic  in 
attempting  to  match  a  target  stimulus  figure  with  one  of  five  alternatives. 

Obstacles  and  Remedies  (PE7933/7934)  --  an  80-Item  speeded  test  that  measures  the 
subject's  ability  to  follow  a  set  of  procedures.  The  subject's  task  Is  to  apply  a  set 
of  rules  and  priorities  In  removing  sets  of  obstacles  encountered  along  an  imaginary 
path. 

Simulated  Navigation  Mission  (PE7936/7938)  —  a  test  that  requires  the  subject  to  plot 
the  location  of  an  aircraft  at  10  points  during  a  "mission."  Radio  s  gnals  generated  at 
two  locations,  flying  time,  wind  speed,  wind  direction,  and  altitude  are  provided  to  the 
subject  for  use  In  calculating  aircraft  locations.  At  each  location,  the  subject  must 
compute  both  the  distance  travelled  since  the  last  reading  and  the  aircraft  speed,  as 
well  as  answer  questions  about  fuel  consumption. 

Rotated  Letters  (PE7932)  --  a  speeded  test  in  which  the  subject  must  judge  whether  each 
of  288  pairs  of  rotated  letters  Is  In  the  same  orientation  or  whether  they  are  mirror 
Images. 

Descriptive  statistics  for  all  of  the  predictor  variables  are  presented  in  Table  1. 


Criterion  Variables 

Two  types  of  variables  were  used  as  indicators  of  UNT  performance.  The  first  was  a  single 
dichotomous  measure  of  overall  training  outcome,  scored  0  for  elimination,  1  for  graduation.  The 
second  type  of  Indicator  consisted  of  summary  variables  for  each  of  the  three  phases  of  training 
( 1 . e . ,  classroom,  simulator,  and  flying  lessons).  These  summary  variables  were  formed  by 
averaging  the  Individual  lesson  grades  for  each  phase  of  training.  Descriptive  statistics  for 
these  summary  scores  and  the  individual  lesson  grades  are  provided  in  Table 


Table  1.  Descriptive  Statistics  for  Predictor  Variables 


Mean  SD  N 


AFOQT  Conposltes 


1. 

Pilot 

61.33 

22.75 

544 

2. 

Navigator-Technical 

56.07 

22.55 

544 

3. 

Academic  Aptitude 

64.17 

22.  71 

544 

4. 

Verbal 

23.84 

544 

5. 

Quantitative 

62.45 

22.52 

543 

Basic  Navigator  Battery  Composites 

1. 

Pre-Nav 

69.34 

15.03 

544 

2. 

Information  Processing 

54.72 

17.29 

544 

3. 

Obstacles  and  Remedies 

58.38 

18.51 

543 

4. 

Simulated  Navigation  Mission 

50.86 

22.10 

544 

5. 

Rotated  Letters 

68.93 

15.67 

542 

Table  2.  Descriptive  Statistics  for  Criterion  Variables 
(Individual  Lesson  Grades  and  Composite  Lesson  Grades) 

Mean 

SD 

N 

UNT  Lessons 

1.  Aerospace  Physiology 

92.57 

5.58 

544 

2.  Airmanship 

93.34 

6.18 

544 

3.  Advanced  Airmanship 

89.31 

7.  50 

541 

4.  Weather 

93.16 

7.09 

540 

5.  Navigation  Procedures 

88.70 

538 

6.  Basic  Navigation 

93.03 

6.27 

509 

7.  Day  Celestial 

85.08 

11.64 

487 

8.  Night  Celestial 

92.18 

6.80 

469 

UNT  Simulator  Lessons 

9.  Navigation  Procedures  T-45 

86.81 

13.95 

527 

10.  Basic  Navigation  T-45 

81.05 

12.68 

505 

11.  Tactical  Navigation  T-45 

88.31 

9.41 

456 

UNT  Flying  Lessons 

1 2.  Radar  T-43 

85.42 

10.  52 

493 

13.  Day  Celestial  T-43 

84.59 

10.  75 

475 

14.  Night  Celestial  T-43 

86.39 

10.39 

467 

15.  Tactical  Navigation  T-37 

89.43 

8.65 

460 

UNT  Graduatlon/El 1*1  nation® 

.84 

.37 

544 

UNT  Composite  Scores 

1 .  Cl assroom  Lessons 

90.77 

5.14 

538 

2.  Simulator  Lessons 

85.29 

8.52 

505 

3.  Flying  Lessons 

86.62 

5.67 

467 

aCoded  1  If  graduate;  0  if  ellmlnee. 
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The  data  were  analyzed  with  hierarchical  multiple  regression  procedures.  A  separate  analysis 
was  conducted  for  each  of  the  four  measures  of  UNT  performance  (i.e.,  overall  training  outcome, 
classroom  grades,  simulator  grades,  flying  grades).  Entry  order  of  the  predictor  variables  In 
each  regression  was  determined  as  follows.  The  Nav-Tech  composite  of  the  AFOQT  was  entered  on 
the  first  step  of  each  regression,  since  It  was  used  to  select  candidates  for  UNI.  The  four 
remaining  AFOQT  composites  were  then  analyzed  to  determine  If  they  Individually  added  to  the 
predictive  validity  of  the  Nav-Tech  composite.  The  significance  of  the  contribution  of  each  of 
the  four  AFOQT  composites  to  the  Nav-Tech-based  prediction  equation  was  assessed  with  F-tests. 
Those  composites  which  added  significantly  to  the  amount  of  criterion  variance  accounted  for  were 
subsequently  entered  Into  an  AFOQT-based  prediction  equation  in  descending  order  of  the  magnitude 
of  their  contribution. 

After  delineating  the  final  AFOQT-based  prediction  equation,  the  five  subtests  of  the  BNB 
were  assessed  for  their  potential  contributions  toward  Increasing  the  amount  of  variance 
accounted  for  In  each  of  the  UNT  performance  measures.  As  in  the  preceding  analysis,  F-tests 
were  used  to  determine  the  significance  of  the  Increase  In  predictive  validity  resulting  from  the 
addition  of  each  BNB  subtest  to  the  AFOQT  equation.  Those  subtests  which  produced  a  significant 
Increase  were  entered  Into  a  final  prediction  equation,  again  in  descending  order  of  the  Increase 
produced. 

It  should  be  noted  that  the  correlations  used  in  the  regression  analyses  were  not  corrected 
for  the  restriction  In  test  score  variance  which  Is  likely  to  result  from  the  use  of  a  sample 
that  Is  pre-selected  on  one  of  the  predictors  of  interest  (i.e.,  AFOQT  Nav-Tech  scores  in  the 
present  study).  Since  this  study  used  a  design  In  which  all  subjects  had  been  pre-selected  for 
UNT  prior  to  testing  on  the  BNB,  It  was  to  be  expected  that  the  magnitude  of  the  validity 
coefficients  would  be  attenuated  to  a  certain  degree.  Thus,  it  Is  reasonable  to  assume  that  1r.  a 
sample  unrestricted  by  pre-selection  effects,  the  strength  of  the  resulting  validity  coefficients 
would  be  larger  than  those  evidenced  In  the  present  study.  However,  the  procedures  used  in  this 
study  do  address  the  primary  concern;  I.e.,  whether  or  not  the  BNB  has  the  potential  to  improve 
the  selection  of  UNT  candidates  over  and  above  that  which  can  be  expected  with  the  AFOQT  already 
In  use. 


III.  RESULTS  AND  DISCUSSION 

The  zero-order  correlations  among  all  predictor  and  criterion  variables  are  presented  In 
Table  3.  All  of  the  AFOQT  and  BNB  subtests,  with  the  exception  of  the  BNB  Rotated  Letters 
subtest,  were  found  to  correlate  significantly  (p  <  .05)  with  the  graduation/el Imlnatlon 
(grad/ellm)  performance  measure.  Similar  results  were  obtained  for  the  classroom  lesson  grades 
criterion.  All  AFOQT  composites  and  BNB  subtests  correlated  significantly  (p  <  .05)  with 
simulator  lesson  grades,  and  all  but  the  AFOQT  Verbal  composite  correlated  significantly  with  the 
flying  grades  criterion. 

The  results  of  the  hierarchical  regression  analyses  are  presented  In  Tables  4  through  7. 
Overall,  these  tables  provide  evidence  that  the  validity  of  the  AFOQT  Nav-Tech  composite  can  be 
laproved  significantly  by  Including  scores  from  selected  AFOQT  composites  and  BNB  subtests  In  the 
selection  decision. 

Table  4  summarizes  the  results  for  the  prediction  of  the  overall  training  outcome.  This 
table  shows  that  adding  three  predictors  (AFOQT  Quantitative,  BNB  Pre-Nav,  BNB  Obstacles  and 


Table  3.  Zero-Order  Correlations  Among  Predictor  and  Criterion  Variables 


1 

2 

3 

4 

5 

6 

*7  ' 

8 

9 

10  11 

12  13 

AFOQT 

1.  Pilot 

- 

2.  Nav-Tech 

73 

- 

3.  Academic 

37 

51 

- 

4.  Verbal 

22 

34 

83 

- 

5.  Quantitative 

38 

58 

76 

38 

- 

BNB 

6.  Pre-Nav 

7.  Information 

47 

53 

53 

36 

56 

Processing 

37 

46 

31 

27 

31 

43 

- 

8.  Obstacles  and 

Remedies 

24 

35 

18 

10 

26 

30 

19 

- 

9.  Simulated  Navi- 

gation  Mission 

39 

51 

44 

29 

50 

53 

33 

34 

- 

10.  Rotated  Letters 

23 

32 

10 

08 

13 

23 

26 

31 

20 

- 

11.  Grad/El  im 

12.  UNT  Classroom 

16 

17 

16 

10 

19 

25 

14 

17 

21 

02x 

Lessons  Grade 

30 

37 

36 

22 

40 

49 

25 

23 

37 

* 

13.  UNT  Simulator 

Lessons  Grade 

26 

29 

24 

11 

32 

42 

21 

26 

33 

14  50 

51 

14.  UNT  Flying 

Lessons  Grade 

21 

24 

09* 

01  a 

14 

27 

18 

17 

22 

09*  20 

33  4? 

Note.  Decimal  points 

omi tted. 

For  each  correlation 

,  n 

varied 

from 

536-547.  All 

correlations 

significant  for  £  <  . 

01  unless 

otherwi se 

noted. 

Nonsignificant  correlation. 
*£  <  .05. 


Table  4.  Prediction  of  Graduation/El imlnatlon  from  UNT 
Predictors 

AFOOT  Nav-Tech  .15  .023 

AFOQT  Nav-Tech  * 

AFOQT  Quantitative  .20  .059  .016** 

AFOQT  Nav-Tech  * 

AFOQT  Quantitative  * 

BNB  Pre-Nav  .25  .063  .024** 

AFOQT  Nav-Tech  * 

AFOQT  Quantitative  * 

BNB  Pre-Nav  + 

BNB  Obstacles  and  Remedies 


.27 


.071 


.007* 


Remedies)  to  the  AFOQT  Nav- lech-based  prediction  equation  significantly  raised  the  coefficient  of 
multiple  correlation  (RJ  from  .15  to  .27,  thus  Increasing  the  amount  of  variance  accounted  for 
(R^)  in  the  graduation/el imination  outcome  measure. 


Table  5  summarizes  the  resul 
grades  perf ormance.  As  in  the 
composite  and  both  the  Pre-Nav 
increased  the  predictive  validi 
Letters  subtest  also  provided 
Nav-Tech  composite  alone  had 
prediction  equation  incorporat 
identified  by  this  analysis  had 


ts  of  the  analysis  for  the  prediction  of  the  UNT  classroom  lesson 
analysis  of  the  grad/ellm  outcome,  using  the  AFOQT  Quantitative 
and  Obstacles  and  Remedies  subtests  of  the  BNB  significantly 
ty  provided  solely  by  the  Nav-Tech  composite.  The  BNB  Rotated 
a  significant  increase  in  overall  predictive  validity.  The 
a  correlation  of  .36  with  classroom  performance,  whereas  a 
ing  the  Nav-Tech  composite  plus  the  other  four  predictors 
a  multiple  correlation  of  .54. 


The  results  of  the  'egression  analysis  for  the  prediction  of  the  simulator  lesson  grades  are 
presented  in  Table  6.  Two  AF0Q1  composites  (Quantitative  and  Pilot)  and  three  BNB  subtests 
(Pre-Nav,  Obstacles  and  Remedies,  Simulated  Navigation  Mission)  produced  significant  increases  to 
the  level  of  prediction  provided  solely  by  the  Nav-Tech  composite.  Addition  of  the  two  other 
AFOQT  composites  significantly  raised  the  correlation  from  .29  to  .35.  Adding  the  three  BNB 
subtest  scores  to  the  prediction  equation  further  signif icantly  raised  the  correlation  to  .46. 

Table  7  presents  the  results  of  the  regression  analysis  for  the  prediction  of  flying  lesson 
grades.  The  Nav-Tech  composite  had  a  correlation  of  .24  with  these  grades.  The  addition  of  the 
AFOQT  Verbal  composite  arid  the  RNB  Pre  Nav  subtest  significantly  raised  the  correlation  to  .33. 


Table  5.  Prediction  of  Average  UNT  Classroom  Lesson  Grades 


Predictors  R  R*  R^  Change 


Table  6.  Prediction  of  Average  UNT  Simulator  Lesson  Grades 


IV.  CONCLUSIONS  AND  RECOMMENDATIONS 

The  major  findings  of  this  effort  can  be  summarized  as  follows: 

1.  The  Nav-Tech  composite  of  the  AFOQT  was  a  significant  predictor  for  all  four  UNT 
performance  criteria  examined. 

2.  The  predictive  validity  of  the  Nav-Tech  composite  was  significantly  Increased  for  each 
criterion  measure  by  the  addition  of  at  least  one  other  AFOQT  composite.  The  Quantitative 
composite  provided  a  significant  increase  In  the  predictive  validity  for  three  of  four  criteria 
(l.e.,  grad/elim,  classroom  grades,  simulator  grades). 

3.  The  predictive  validity  of  the  Nav-Tech  and  the  other  selected  AFOQT  composites  was 
significantly  Increased  for  all  criteria  by  the  addition  of  certain  BNB  subtests  to  the 
prediction  equation.  The  Pre-Nav  subtest  produced  a  significant  Increase  In  predictive  validity 
for  all  four  criteria,  while  the  Obstacles  and  Remedies  subtest  significantly  Incremented  the 
predictive  validity  for  three  of  four  criteria  (i.e.,  grad/elim,  classroom  grades,  simulator 
grades) . 

These  results  provide  preliminary  evidence  that  the  predictive  validity  of  the  current  selection 
system  for  UNT  trainees  can  be  Improved  by  adding  Information  from  the  AFOQT  Quantitative 
composite  and  at  least  two  BNB  subtests  to  the  selection  decision  process.  The  practical 
Implications  of  the  potential  Increase  In  predictive  validity  are  illustrated  In  Table  8.  This 
table  shows  the  percentage  of  trainees  In  the  present  sample  which  would  have  been  correctly  or 
Incorrectly  rejected  using  the  three  prediction  models  Identified  In  the  analysis  of  UNT 
graduatl on/elimination.  The  upper  portion  of  the  table  summarizes  the  percentage  of  ellmlnees  In 
the  sample  who  would  have  been  correctly  rejected  (i.e.,  true  negatives)  by  rank-ordering  the 
candidates  by  their  final  prediction  equation  scores  and  then  using  one  of  three  different 
percentile  scores  as  a  cutoff  point.  These  percentages,  therefore,  are  Indicative  of  the 
relative  efficiency  of  each  model  at  correctly  screening  out  unqualified  UNT  candidates.  Model 
I,  containing  only  the  AFOQT  Nav-Tech  composite,  screened  the  lowest  percentage  of  ellmlnees  at 
all  percentile  score  levels.  Model  III,  containing  the  AFOQT  Nav-Tech  and  Quantitative 
composites,  and  BNB  Pre-Nav  and  Obstacles  and  Remedies  scores,  correctly  screened  the  largest 
percentage  of  unqualified  candidates  at  all  the  percentile  scores. 

The  lower  portion  of  Table  8  provides  data  on  the  percentage  of  UNT  graduates  who  would  have 
been  incorrectly  rejected  (I.e.,  false  positives)  by  each  model  for  three  percentile  cutoff 
scores.  Because  these  percentages  represent  each  model's  efficiency  In  minimizing  the  Incorrect 
rejection  of  qualified  candidates,  smaller  percentages  Indicate  greater  efficiency.  These 
comparisons  revealed  very  little  difference  among  the  three  models  at  any  of  the  percentile 
cutoff  scores.  Essentially,  all  three  models  were  equivalent  In  their  rates  of  incorrectly 
rejecting  candidates  who  would  successfully  complete  UNT.  Overall,  considering  the  correct  and 
Incorrect  rejection  characteristics  of  all  three  models,  Model  III  appears  to  be  the  best 
available  selection  system  at  the  present  time. 

Although  these  results  are  encouraging.  Incorporation  of  the  changes  suggested  by  this  study 
Into  the  present  system  for  selecting  navigator  trainees  would  be  premature  at  this  time. 
Additional  research  on  the  BNB  is  needed  In  two  areas.  First,  the  psychometric  characteristics 
of  the  BNB  subtests,  such  as  Internal  consistency,  should  be  assessed  at  a  more  in-depth  level 
than  was  possible  In  the  present  study  since  only  total  test  scores,  not  individual  Item  data, 
were  available  for  each  subtest.  Second,  the  unique  predictive  validity  of  the  BNB  subtests 
should  be  assessed  when  they  are  given  at  the  same  time  as  the  AFOQT.  In  the  present  study, 
because  of  testing  limitations,  the  BNB  was  given  to  UNT  candidates  1  week  prior  to  their 
training  whereas  the  AFOQT  was  administered,  for  operational  selection  purposes,  several  months 
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prior  to  training.  The  effects  of  this  time  difference  could  not  be  accounted  for  In  the  present 
effort.  A  follow-on  study  with  concurrent  administration  of  the  AFOQT  and  BNB  could  provide  the 
data  necessary  to  address  both  areas  of  concern. 


Three  recommendations  for  future  navigator  selection  research  are  offered.  First,  a  study 
should  be  conducted  In  which  simultaneous  data  collection  on  all  predictor  measures  Is  possible. 
Optimally,  all  tests  would  be  given  to  UNT  applicants  prior  to  their  operational  selection.  This 
type  of  design  would  allow  for  an  assessment  of  the  unique  predictive  validity  of  the  BNB  and 
also  provide  an  estimate  of  the  distribution  of  BNB  test  scores  within  the  applicant  pool. 
However,  because  of  applicant  testing  time  constraints,  this  approach  might  not  be  feasible.  A 
fall-back  approach  would  be  to  re-test  UNT  selectees  on  the  AFOQT  and  administer  the  BNB  just 
prior  to  training.  Although  this  design  would  not  provide  a  good  estimate  of  the  applicant  pool 
BNB  score  distribution.  It  would  answer  the  major  question  about  the  unique  predictive  validity 
of  the  BNB  when  administered  with  the  AFOQT. 


The  second  reconmendatlon  is  to  begin  the  development  of  new  forms  of  the  BNB.  The  current 
version  could  not  be  used  operationally  because  it  has  been  available  for  several  years  as  an 
uncontrolled  test.  Therefore,  It  Is  likely  that  an  unacceptable  level  of  test  compromise  ( 1 . e. , 
applicants  having  access  to  test  questions)  would  prevent  Its  operational  use.  For  operational 
use,  new  versions  would  have  to  be  developed  and  protected  from  compromise  such  that  applicants' 
scores  would  reflect  their  true  abilities  rather  than  their  familiarity  with  the  test. 


The  third  reconmendatlon  Is  more  general.  Future  efforts  In  navigator  selection  research 
might  benefit  from  ongoing  research  In  the  field  of  computerized  selection  testing.  As  part  of  a 
program  to  Improve  pilot  selection,  a  computerized  test  system  has  been  developed  to  measure 
Information  processing  capabll Itles,  psychomotor  abilities,  and  certain  personality 
characteristics.  A  similar  computerized  system,  adapted  specifically  for  navigator  selection, 
might  further  Increase  the  accuracy  of  prediction  over  that  obtainable  from  conventional 
paper-and-pencll  tests  such  as  the  AFOQT  and  BNB. 


V  \  A  *-  *-  *. 
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